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ABSTRACT 



This document reviews studies of professional development 
for, teachers to examine benefits to students in science and mathematics 
education. A major finding from this review was that programs that focused on 
teaching classroom management strategies or attaining knowledge of how 
students learn specific school subject matter provided later benefits to 
students. The more successful professional development programs were not 
simply courses in mathematics or science, but instead were about what to 
teach and how students learn that subject matter. (Contains 17 references.) 
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Benefits of 
teacher professional 
development for 
students have 
too seldom 
been examined. 
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T here is a much-maligned event in educa- 
tion called the one-shot workshop. This 
event has been criticized by virtually 
every teacher who has ever participated in it and 
by virtually everyone else even vaguely interested 
in improving teaching. Researchers and policy 
analysts, critical of the one-shot workshop, have 
generated a number of proposals for how contin- 
uing education programs for teachers should be 
organized (Corcoran, 1995; Goldenberg & 
Gallimore, 1991; Little, 1993; Loiicks-Horsley 
et al., 1998). They argue that O 



• programs should be lengthy rather than brief 

• teachers should have a role in defining the 
content rather than having the topics imposed 
on them 

• the scheduled meetings should be interspersed 
with classroom practice rather than concen- 
trated into a short period of time 

• teachers should work together in groups, 
rather than in isolation 

There is a common-sense appeal to these ideas. 
It makes sense that, if you really want to alter 
teaching practice, you need more than a two-hour 
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workshop. But the ultimate benefits of 
these recommended changes have seldom 
been examined. 

This Brief examines these contentions 
by reviewing studies of professional devel- 
opment that examine benefits to students. 
A major finding from this review is that 
program content — what is being taught, 
such as classroom management strategies 
or knowledge of how students learn 
specific school subject matter — is an 
important predictor of benefit to 
students. This finding should not be a 
surprise, but it is in light of the literature 
describing optimal professional develop- 
ment, which does not address content 
as much as it addresses program form 
and structure. 

The Literature 

A total pool of 93 studies were found that 
examined the effectiveness of various 
approaches to continuing teacher educa- 
tion in either mathematics or science. 
However, only 10 included evidence of 



benefits to students. The paucity of 
evidence for how these programs 
ultimately benefitted students is itself an 
important finding. 

Table 1 groups studies according to 
program content. The four categories 
include, respectively: 

• 7, Programs that prescribe a set of 
teaching behaviors that are expected to 
apply generically to all school subjects? 
These behaviors might result from 
process-product research or might 
include things like cooperative 
grouping. In either case, the methods 
are expected to be equally effective 
across school subjects; 

• 2, Programs that prescribe a set of 
teaching behaviors that seem generic, 
but are proffered as applying to one 
particular school subject, such as math- 
ematics or science. Though presented 
in the context of a particular subject, 
the behaviors themselves have a 
generic quality to them, in that they 
are expected to be generally applica- 



ble across all topics in that subject; 

• 3, Programs that provide general guid- 
ance on both curriculum and pedagogy 
for teaching a particular subject, and 
that justify their recommended prac- 
tices with references to knowledge 
about how students learn this subject, 
and 

• 4, Programs that provide knowledge 
about how students learn particular 
subject matter but do not provide spe- 
cific guidance on the practices that 
should be used to teach that subject. 

The programs being examined in these 
studies also differ along many of the 
dimensions that reformers care about: 
intensity, focus on individual teachers 
versus schoolwide focus and so forth. One 
difference that also needs to be attended 
to, however, is the varying durations of 
the studies. Some followed students for an 
entire school year or longer, while others 
followed them for only a semester or less. 
Longer study durations can reduce appar- 
ent program effects because they increase 



Table 1 . Studies Included in This Review 



CITATION 


SUBiEQMAnER 

CONTEXT 


GRADE SPAN OF 
PARTICIPATING STUDENTS 


SOURCE OF PARTICIPANTS 


FORM AND DISTRIBUTION OF 
CONTAQTIME 


TOTAL CONTAa 
HOURS* 


STUDY DURATION 
IN MONTHS* 
















Category 1 : Content Focus Is on Teaching Behaviors That Apply Generically to Ail School Subjects 


Stallings & Krasavage (1986) 


Math 


2-4 


Schoolwide projects 


Distributed workshops 




16month 


Stevens &Slavin (1995) 


Math 


K-6 


Schoolwide projects 


Distributed workshops 




8 months . 


Category 2: Content Focus Is on Teaching Behaviors That Apply to a Particular Subject 


Good, GrouwS;& Ebmeier (1983) 


Math 


4-12 


Individual volunteers 


2 @1.5 


3 


4 


Good &Grouws (1979) 


Math 


4 


Individual volunteers 


2 @1.5 


3 


4 


Mason & Good (1993) 


Math 


4-6 


Individual volunteers 


3 @1.5 


4.5 


5 


Otto &Schuck (1983) 


Science 


8 


Individual volunteers 


5 @ variable 


16 


2.5 


Rubin & Norman (1992) 


Science 


6-9 


Individual volunteers 


University course (10 @3) 


30 


3 


Lawrenz & McCreath (1988) 


Science 


1-8 


Individual volunteers 


University course (15 @3) 


45 


8 


Marek & Methven (1991) 


Science 


1-5 


Individual volunteers 


4-week Summer Institute 


100 


8 


Category 3: Content Focus Is on Curriculum or Pedagogy Justified by How Students Learn 


Cobb etal. (1991) 


Math 


2 


Individual volunteers 


1-week Sum. Inst.-i- Distributed 


150 


8 


Wood & Sellers (1996) 


Math 


2-3 


Individual volunteers 


1-week Sum. Inst, -i- Distributed 


150 


16 


Category 4: Content Focus Is on How Students Learn and How to Assess Student Learning 


Carpenter etal. (1989) 


Math 


1 


Individual volunteers 


4-week Summer Institute 


80 


8 



NOTE: Studies reviewed are listed at the end of this Brief. 

* Some of these estimates of contact time had to be estimated from general descriptions of programs, for estimates of program durations, we assumed a "school year" was roughly 8 months, a semester was 4 months, and two school years were 16 months. 
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the likelihood that other events' (staffing 
changes, fire drills or other traumas, other 
policy changes, etc.) will disrupt program 
influences. Thus as we examine the findings 
from these studies, we need to be wary of 
findings that are based on short-term studies: 
Their program effects may appear larger than 
those of longer-term studies, not because of 
differences in program quality but instead 
because of differences in study length. 

One difference among these categories is 
especially important, and that is their tacit 
model for how they expect their programs to 
eventually influence student achievement. 
Underlying these different approaches to 
continuing professional education are differ- 
ent assumptions about the path between the 
program and its eventual effects on student 
learning. Figure 1 illustrates these differing 
sets of assumptions. Programs in Categories 
1 and 2 expect their programs first to change 
teacher behaviors, and expect that these 
behavioral changes will, in turn, lead to 
student learning. Those in Categories 3 and 



4, on the other hand, expect their programs 
to first change teacher knowledge', they tend 
to be relatively less prescriptive about 
teaching practices. The Category 3 programs 
provide teachers with knowledge about 
how students learn mathematics, with some 
curriculum materials, and with some ideas 
about new practices that will better promote 
student learning. The program in Category 
4 focuses even more narrowly on teacher 
knowledge, specifically knowledge of 
how students learn particular mathematical 
ideas. These program developers do, of 
course, expect teaching practice to change, 
but instead of prescribing the details of the 
new practice, they assume that changes in 
teacher knowledge will stimulate teachers to 
devise their own new teaching practices that 
will, in turn, lead to student learning. 

These four categories of program 
content, then, reflect continua from more 
prescriptive to more discretionary, and from 
more focused on behavior to more focused 
on ideas. 



Programs Aimed at 
Improving Student Learning 
in Mathematics 

For both of the illustrative studies in 
Category 1, contact time was extensive and 
distributed throughout the school year, 
teachers received in-class visitations, and the 
programs worked with whole schools rather 
than individual volunteers. Thus, these pro- 
grams represent the kind of professional 
development that has been recommended. 

The Category 2 studies focusing on 
mathematics all addressed a single program, 
the Missouri Mathematics Model sponsored 
by Tom Good and his colleagues (Good & 
Grouws, 1979; Good, Grouws, & Ebmeier, 
1983; Mason & Good, 1993). These 
programs are typically very brief, consisting 
of just two 90-minute sessions during which 
the specific recommended teaching behav- 
iors and their rationales are explained. 
Teachers also receive a manual with more 
detailed discussion of the Missouri 
Mathematics Model. 

The programs in Categories 3 and 4 
differ considerably from those in Category 2 
and are similar to one another in their theo- 
retical orientations. These programs are 
interested in student cognition, both assume 
some form of constructivist theory of learn- 
ing, and both are interested in increasing 
teachers' attention to problem solving and 
reasoning in place of recall of computational 
procedures. Both Category 3 studies focus 
on a single program, the Problem-Centered 
Mathematics Program. This program 
provides teachers with knowledge about 
student learning and thinking in mathemat- 
ics, gives them mathematics problems that • 
are designed to be challenging for students 
at the grade level they teach, and gives them 
a class discussion format that encourages 
thoughtful engagement with these problems. 
The one study in Category 4 examines the 
Cognitively Guided Instruction program, 
which is similar to the Problem-Centered 
Mathematics Program in its orientation to 



mathematics, but focused more on the 
particular mathematical content that 
students learn in the relevant grade levels 
and on the particular kinds of difficulties 
they are likely to have in learning this 
content. The Category 4 study does 
not define \vhat teachers should do v^ith 
this knowledge. 

Table 2 shows the size of program 
effects on student achievement in mathe- 
matics that were found in each category. 
Each number indicates the size of the 
program effect in standardized units rela- 
tive to a comparison group. With the 
exception of Category 1 programs, which 
worked with whole schools, all studies 
involved teachers who volunteered to par- 
ticipate and who were randomly assigned 
to experimental and comparison groups. 

Researchers tended to measure three 
types of effects: basic skills, advanced rea- 
soning, and attitudes toward the subject. 
Basic skills were generally assessed with 
traditional standardized achievement tests, 
and researchers devised their own proce- 
dures for assessing advanced reasoning and 
attitudes. Table 2 suggests that programs 
in Categories 3 and 4 demonstrate greater 
gains in reasoning and problem solving as 
well as comparable or greater gains 
in basic skills. Even in basic skills, 
the smallest program effects were in 
Category 1 and the largest in Category 4. 
This pattern of outcomes suggests that the 
content of programs does indeed make a 




Promoting teacher knowledge about how 
students learn particular concepts^ skillsy 
and applications is especially powerful. 



difference, and that programs that focus 
on subject matter knowledge and on 
student learning of particular subject 
matter are likely to have larger positive 
benefits for student learning than pro- 
grams that focus mainly on teaching 
behaviors. This pattern is particularly 
striking in light of the fact that the two 
programs in Category 1 more closely 
approximate the ideal in terms of form 
and structure than do the programs in 
Categories 3 and 4. 

Why do the Category 3 and 4 pro- 
grams have greater effects on students? 



Several hypotheses have been suggested. 
One early hypothesis was that teachers in 
Categories 1 and 2 could not improve 
their mathematics teaching because they 
did not have adequate subject matter 
knowledge. However, the more successful 
programs in this review were not provid- 
ing subject matter knowledge per se, but 
rather knowledge about how students 
learn subject matter knowledge. No doubt 
teachers acquired some subject matter 
knowledge along the way in these 
programs, but teacher knowledge was 
not the central focus of programs in 
either Category 3 or Category 4. 

Another hypothesis is that, by giving 
teachers a greater understanding of how 
students learn, programs in Categories 3 
and 4 enable teachers to continue to 
develop and refine their own practices. 
That is, it is the lack of prescriptiveness 
that makes this knowledge valuable. 
In contrast, the Madeline Hunter 
program (Category 1) and the Missouri 
Mathematics Model (Category 2) 
both prescribe virtually invariant daily 
routines. Though not so rigid, there is also 
a recommended pattern for classroom 
activities and a recommended set of learn- 
ing activities in the Problem Centered 
Mathematics Program (Category 3) as 
well. The Cognitively Guided Instruction 
program provided teachers with the least 
amount of specific information about 
what they should do in their classrooms 



Table 2. Tables. 

Average Standardized Effect Sizes Achieved in Mathematics Standardized Effect Sizes Achieved in Science 



CATEGORY 


BASiC SKILLS 


REASONING, 

PROBLEMSOLVING 


AHITUDES 
-TOWARD MATH 


Category 1 


-.14 


.10 




Category 2 


.17 


.05 




Category 3 


.13 


.50 


.13 


Category 4 


.52 


.40 





CATEGORY 2 
CONTENT FOCUS 


BASiC SKILLS 


REASONING, 

PROBLEMSOLVING 


AHITUDES 
TOWARD MATH 


Category 2: Content Focus Is 


on Teaching Behavi 


ors That Apply to a 1 


Particular Subject 


(a) Modeling as a 
Teaching Strategy 

(b) Learning tole as a 
Teaching Strategy 




.71 

.43 


.15 
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The format of professional development appears to be less important than the content 



and with the most specific information 
about the mathematics content they 
would be teaching and how students learn 
that content. 

Programs Aimed at 
Improving Student 
Learning in Science 

The four science studies that provided 
student outcome data fell entirely into 
Category 2: They claim to offer teachers 
techniques that are uniquely suited 
to science teaching, but the techniques 
themselves still have a generic character, in 
that they do not depend on the particular 
science content being taught. For instance, 
Rubin and Norman taught teachers to 
model particular science processes such as 
generating hypotheses, identifying and 
controlling variables, and defining things 
operationally. During their program, they 
used generic lesson formats to train 
teachers to model each of these skills. 
Modeling the skill of "identifying and 
controlling variables" for example, consists 
of asking aloud such questions as, "What 
is the manipulated variable in this 
experimental situation?" 

' However, the Category 2 studies in 
science reflect two different models of 
teaching. To reflect this difference in 
program content. Table 3 provides two 
subgroupings within these programs. 

The effects shown in Table 3 are larger 
than their counterparts in Table 2, a 
difference that probably reflects greater 
alignment between instructional content 
and assessment content. Because science 
curricula in American schools are less 
standardized than mathematics curricula, 
and because science content is not 
normally included in standardized 
achievement tests, science researchers are 
more likely to devise their own curriculum 
materials and their own outcome mea- 
sures, as was the case in these studies. 
Consequently, there is likely to be a 
greater articulation between the content 







taught in participating "treatment" class- 
rooms and the content assessed by the 
science researchers than is the case in 
mathematics programs. 

Like Table 2, Table 3 appears to suggest 
that program content matters. It suggests 
that programs focusing on scientific 
processes had greater effects than those 
focusing on the learning cycle. However, 
the two studies that taught teachers to 
model scientific processes were extremely 
brief, extending only Th and 3 months, 
respectively, whereas the two studies that 
taught students the learning cycle 
were full-year studies. Consequently, 
differences in effects that appear to reflect 
program content differences could be a 
function of study duration rather than 
program content. 

The Relevance of Program 
Form and Structure 

The programs examined in this small 
body of research represent a variety of 
program structures, and this variety 
enables us to examine the merits of several 
hypotheses about critical features of 
continuing professional education. The 

5 . 0 



patterns suggest that program content is a 
central predictor of benefit to students. 
They also suggest that many other 
program dimensions are less reliable 
producers of benefits for students. Briefly, 
in this small sample of studies, we can 
conclude the following: 

• Differences in total contact hours 
were unrelated to student outcomes. 
Programs in Category 1 provided far 
more contact hours than programs in 
Category 2, and yet had smaller 
effects on student learning. Similarly, 
the Category 3 programs provided 
more contact hours than the 
Category 4 program did, and yet 
did not yield noticeable advantages 
for students. 

• Evidence was mixed for the benefits 
of distributed time. The studies in 
mathematics did not support this 
hypothesis, for the mathematics 
program with the most substantial 
overall influences on student learning 
consisted of a summer institute with 
no distributed seminars during the 
next academic year. Conversely, the 
one program that demonstrated 




negative effects on student learning, 
the Madeline Hunter program 
studied by Stallings and Krasavage, 
provided both seminars and in-class 
visitations throughout the school 
year. Studies in science, on the other 
hand, offer some support: One of the 
studies that focused on the learning 
cycle provided a concentrated 
summer institute, while the other 
provided a university course with 
sessions distributed across a full 
school semester. The distributed 
program appeared to produce greater 
benefits to students than did the 
concentrated summer institute. 

• None of the programs that provided 
in-class visitations produced notice- 
ably greater benefits to student 
learning. 

• The fact that the Category 1 
programs — those working with whole 
schools — demonstrated the smallest 
influences on student learning among 
these studies suggests that providing 
services to whole school staffs may 
not be the most important feature 
of continuing professional education 
for teachers. However, it is likely 
that whole school programs involve 
at least some teachers who did 
not volunteer to participate, and this 
fact may reduce the apparent 
program benefits. 

Summary and Conclusion 

Based on the studies reviewed here, a 
strong case can be made for attending 
more to the content of continuing 
professional education and for attending 
less to the structural and organizational 
features of such programs. In these 
studies, programs whose content focused 
mainly on teachers* behaviors demonstrat- 
ed smaller influences on student learning 
than did programs whose content focused 
on how students learn particular subject 




Focusing on teacher behaviors has 
smaller influences on student learning 
than focusing on how students learn 
particular subject matter. 



matter. The more successful professional 
development programs were not simply 
courses in mathematics or science, but 
instead were about what to teach and how 
students learn that subject matter. Cohen 
and Hill (1998), in their study of 
California mathematics reform, also find 
that the content of professional develop- 
ment is important. The programs in 
Categories 3 and 4 were very specific in 
their focus. They did not address generic 
learning, but instead addressed the learn- 
ing of particular mathematical ideas. 

An equally important finding from this 
review is the lack of clear benefit of several 
popular structural program features. The 
programs reviewed here differed in the 
total number of contact hours they 
provided teachers, in whether or how that 
time was distributed, in whether that time 
included in-class visitations, and in 
whether teachers participated as members 



of whole schools or as individuals. The 
reason for the lack of clear benefit 
for these program dimensions is likely 
related to the important role of program 
content: A program whose content is 
not valuable will not be improved by 
increasing the number of contact hours, 
distributing contact hours over time, 
providing in-class visits, and so forth. 
Structural features alone provide no 
guarantee of improved teacher learning or 
of eventual benefit to students. What is 
still unclear, however, is whether, given 
important content, these structural 
features of programs might further 
enhance benefit to students. The central 
message from these studies, though, is 
to attend to content first, before attending 
to structure. 

The findings presented here suggest 
that the content of professional develop- 
ment, should be attended to first, before 
form and structure. The findings also 
suggest that effective professional 
development in mathematics and science 
treats teachers as professionals. Reform 
advocates have argued that teachers will 
profit more from knowledge and insights 
they develop in their own ways, rather 
than from prescriptions that give them 
little practical leeway. The pattern of 
program effects shown here suggests that 
these reformers are right. 



For more information contact 
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ENDNOTES 



STUDIES REVIEWED 



^ This Brief is a summary of material in a research monograph by Mary 
Kennedy (1998), Form and substance in inservice teacher education (Research 
Monograph No. 13). Madison: University of Wisconsin-Madison, National 
Institute for Science Education. For ordering information contact 
NISE at (608) 263-9250, e-mail: niseinfo@mail.soemadison.edu, 
Web site: wvw.nise.org 

^ Technically, Category 1 programs are not aimed specifically at mathe- 
matics or science education but instead offer a set of ideas that are presumed 
to be applicable to all school subjects. However, because such programs con- 
stitute a large fraction of professional development, and because such 
programs typically include mathematics test scores in their portfolio of out- 
comes, two studies that illustrate this line of work are included. 
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